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African-American elementary students in grades 3 through 6, Three 
factor analyses were conducted to determine a general pattern among 
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the overall results for grade 3 students and grade A through 6 
students. The results give evidence supporting the belief that the 
standardized test and portfolio measure different constructs and that 
both forms of assessment exert an influence on classroom assessment, 
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Abstract 

If teachers are teaching a set of standard contei.'t and assessments are made consistently, 
the relationship between these various assessments and the curriculum should be of interest. The 
purpose of this study was to explore the consistency of the mathematical measures of three 
prominent forms of mathematics assessment: standardized tests, portfolios, and classroom 
grades. The sample used in this study was a nonrandom sample of 50 African-American 
elementary students in grades 3 through 6. Three factor analyses were conducted to determine a 
general pattern among the variables for the overall sample and explore the consistency of the 
overall results for grade 3 students and grade 4 through 6 students. The results give evidence 
supporting the belief that the standardized test and portfolio measure different constructs and 



that both forms of assessment exert an influence on classroom assessment. 
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Introduction 

As part of a Magnet Assistance School Program, students in grades three through six 
were provided mathematics instruction according to the latest NCTM standards with emphasis 
on solving real-life problems. After much discussion, the researchers concluded that teachers 
were teaching a set of standard concepts as outlined in the school district's mathematics 
curriculum and that the students were given three different assessments: a national standardized 
test, a portfolio assessment, and regular classroom quarterly grades. If teachers are teaching a 
set of standard content and assessments are made consistently, the relationship between these 
various assessments and the curriculum should be of interest. The purpose of this study was to 
explore the consistency of the mathematical measures of these three prominent forms of 
mathematics assessment. 



Methodology 



Sample 

The sample used in this study was a nonrandom sample of 50 African-American 
elementary students in grades 3 through 6. The sample consisted of 19 males and 31 females. Of 
the 50 students, 22 were in grade 3, iO were in grade 4, 14 were in grade 5, and 4 were in grade 
6. All of the students in the sample were enrolled in a magnet school program located within a 
traditional school setting. The program was centered around a multi-cultural theme and was 
designed to make extensive use of multimedia technology. 
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Dependent Variables 

CTB McGraw Hill Mathematics Portfolio . The CTB Portfolio Assessment is designed to 
offer school districts access to an easily administered mathematics assessment for grades 3 
through 8. The assessment is designed to divide the school year into four equal periods. At the 
end of each period, students are assigned performance activities that require them to integrate 
their mathematics knowledge in solving real-life problems. Like traditional portfolios, students 
accumulate their work in portfolio folders that can be kept in the classroom. The CTB Portfolio 
Assessment includes scoring guidelines that are unique to each activity (“CTB Portfolio”, 1992). 
However, for this study, the scoring was slightly modified an^ results were reported as the 
percent correct. Additionally, the assessment was only administered during the last nine week 
period of the school year. 

Stanford Achievement Test . The Stanford Achievement Test (SAT), Eighth Edition is a 
national standardized test that is “designed to reflect what is being taught in schools throughout 
the country.” (“Stanford Achievement”, 1990, p. 1 1) The test was administered during the third 
nine weeks and the score used in this analysis was the national percentile for each student on the 
mathematics subtest. 

Classroom Grades . Classroom assessment was measured using the teacher reported 
grades for each of the four nine week periods in the 1993-1994 academic year. 

Procedure 

The data used in this study came from a larger study i' /olving two magnet school 
programs. The data for the CTB Mathematics Portfolio was furnished by the school district in 
the form of computer printouts. The Stanford Achievement Test percentile scores were also 
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furnished by the district via diskette. The data was confirmed using computer printouts since 
students received different identification numbers and only fourteen of the students were able to 
be matched on a computer using their identification numbers. The classroom grades were 
collected by the researchers with cooperation from the participating school. 

Method 

In order to determine whether the measures addressed the same constructs, three separate 
factor analyses were conducted. The first analysis was conducted using the entire sample of 50 
students. This analysis was used to determine a general pattern among the assessments. The 
remaining two analyses explored the patterns for grade 3 students and grade 4 through 6 students 
separately in order to determine the consistency of the overall factor pattern. For each analysis, 
a principal components technique was used to extract factors, and the number of factDrs 
extracted was determined using the eigenvalue-one criterion. When applicable, the initial factor 
loading matrices v.ere rotated using the Varimax rotation technique in order to aid in the 
interpretation of the factors. Using criteria contained in Hair, Anderson, Tatham, and Black 
(1995) factor loadings larger than +/- 0.50 were considered practically significant. 

Results and Discussion 
Analyses Involving the Overall Sample 

The correlations between each of the mathematics measures are presented in Table 1 Of 
the fifteen computed correlations, nine were statistically significant at the .05 level. The largest 
correlation was between the first nine weeks average and the second nine weeks average, r = .71, 
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< .05. The weakest correlation was between the Stanford Achievement Test and the CTB 
Mathematics Portfolio, r = . 12, £ > .05. 



Insert Table 1 About Here 



The unrotated factor solution for the analysis involving the overall sample is contained in 
Table 2, The table shows that the strongest associations with factor 1 occurred with the first and 
second nine weeks average. Additionally, the strongest associations with factor 2 occurred with 
the third and fourth nine weeks average. Table 2 also provides the final communality estimates 
for each of the assessments. These values range from a high of .77 for the third nine weeks 
average to a low of .36 for the Stanford Achievement Test. Finally, the table shows that factor 1 
accounts for 41.2% of the variance while factor 2 accounts for an additional 22.5%. 



Insert Table 2 About Here 



Since two factors were extracted in this analysis, the Varimax rotation technique was 
applied to the unrotated factor matrix. The results of the rotation technique are contained in 
Table 3. The results show that four of the six measurements weight significantly on one factor 
and low on the remaining factor. Although the assessments for the first and second nine weeks 
weight significantly on factor 1, .65 and .70, respectively, they also provide weight to factor 2, 
48 and .51. respectively. 



Insert Table 3 About Here 
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When viewing the rotated factor loadings for the Stanford Achievement Test and the 
CTB Mathematics Portfolio, it appears that factor 1 represents mathematical knowledge as 
measured by the Stanford Achievement Test. Similarly factor 2 appears to represent 
mathematical knowledge defined by the CTB Mathematics Portfolio. Further examination of the 
quarterly classroom grades shows that the association between the classroom grades and factor 1 
increases from the first to third quarter before decreasing in the fourth quarter. The factor 
loadings for the classroom grades on factor 2 are similar to those for factor 1. There is an 
increase in association from the first to the second quarter. However, the association between 
the third quarter and factor 2 decreases substantially before returning to a significant association 
in the fourth quarter. 

Analyses for Students in Grade 3 

Table 4 contains the pairwise correlations between each of the mathematics measures. 
The table shows that thirteen of the fifteen correlations were statistically significant at the .05 
level. The largest correlation was between the first nine weeks average and the second nine 
weeks average, r = .80, £ < .05. The weakest correlation was between the second nine weeks 
average and the Stanford Achievement Test, r = .21, £> .05. 



Insert Table 4 About Here 



Table 5 contains the unrotated factor matrix and communality estimates for the analysis 
of students in grade 3. In this analysis only one factor was extracted using the eigenvalue-one 
criterion. The loadings on the factor range from a maximum of .86 for the first nine weeks 
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average to a low of .59 for the Stanford Achievement Test. The communality estimates ranged 
from a high of .73 for the first nine weeks average to a low of .35 for the Stanford Achievement 
Test. The table also shows that the single factor accounted for 59.6% of the variability. Because 
only one factor was extracted, the Varimax rotation was not applied. 



Insert Table 5 About Here 



Analyses for Students in Grades 4 through 6 

The correlations between the mathematics measures for students in grades 4 through 6 
are contained in Table 6. The table illustrates that eleven of the fifteen pairwise correlations 
were statistically significant at the .05 level. The strongest correlation was between the first and 
second nine weeks averages, r = 64, < .05. The weakest correlation was between the first nine 

w'eeks average and the Stanford Achievement Test, r = -.02, g < .05. 



Insert Table 6 About Here 



Table 7 contains the unrotated factor solution and communalities for the analysis 
involving students in grades 4 through 6. The table shows that two factors were extracted using 
the same criterion as the previous two analyses. The unrotated loadings for factor 1 range from 
a high of .84 for the first nine weeks average to a low of . 1 1 for the Stanford Achievement Test. 
For factor 2, the loadings range from a high of .92 for the third nine weeks average to . 18 for the 
first nine weeks average. The table also shows that the communalities range from a high of .86 
for the third nine weeks average to a low of 38 for the Stanford Achievement Test and that 
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factor 1 accounts for 38.3% of the variability while factor 2 accounts for an additional 31.8% of 
the variability. 



Insert Table 7 About Here 

The results of the Varimax rotation technique are contained in Table 8. The table shows 
that four of the six variables show a significant association with Oi..y one of the two factors. The 
two remaining variables, first and second nine weeks averages, do show significant associations 
with factor 1, .78 and .71, respectively; however, they also show associations with factor 2, .36 
and .58, respectively. 



Insert Table 8 About Here 



The high rotated factor loading for the CTB Mathematics Portfolio and the low factor 
loading for the Stanford Achievement Test on factor 1 suggests that this factor represents 
constructs that are measured by the CTB Mathematics Portfolio but not the Stanford 
Achievement Test. Likewise, the factor loadings of the two variables on factor 2 suggest that 
this factor represents constructs that are associated with the Stanford Achievement Test opposed 
to the CTB Mathematics Portfolio. An examination of factor 1 in terms of the quarterly 
classroom assessments shows significant associations for quarters 1, 2, and 4 with only a slight 
decrease in size from quarters 1 to 2. Despite the strong associations of the first two and last 
quarters, the association between quarter 3 and factor 1 is nearly zero. 
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The factor loadings for the classroom assessment on factor 2 also show a noticeable 
pattern. There appears to be an increasing trend from quarter 1 to quarter 3, However, the trend 
ends at quarter 3, and the magnitude of the loading drops drastically during quarter 4, 

In summary, the only analysis that did not yield two factors was the grade 3 sample 
indicating that for grade 3 the three forms of assessment are aligned and attempt to measure the 
same construct. However, the results of the remaining two analyses showed that the three forms 
of assessment represented two underlying constructs. One construct appeared to be related to 
assessment defined by the Stanford Achievement Test while the other construct was related to 
assessment in terms of the CTB Mathematics Portfolio. Additionally, the percent of variance 
accounted for by each of the factors indicates that neither factor served as a predominant 
underlying construct. 

The results also indicate that classroom assessment in general was related to both factors 
supporting a statement by the Mathematical Sciences Education Board (1993) who said that 
teachers may attempt to prepare students for tests while simultaneously offering a richer and 
more challenging curriculum. Although classroom assessment was related to both factors, 
distinct patterns among the classroom assessments can be identified. In both analyses, the 
association between classroom assessment and the factor representing the Stanford Achievement 
Test increased until quarter three, when the Stanford Achievement Test was administered. 

Conclusions 

There are several limitations involved in this study that must be considered when viewing 
the conclusions First, the sample involved in this study was nonrandom which affects the 
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generalizability of the results. Therefore, they should be viewed as a description of the 
consistency and relationships that existed for this sample of students. The second limitation 
concerns the reliability and validity of the dependent variables, particularly the CTB 
Mathematics Portfolio since the scoring was modified. Third, the sample sizes used for the 
grade analyses were below the five subjects per variable minimum suggested by Hair et al. 
However, the overall sample was adequate using the same criteria. Finally, because the CTB 
Mathematics Portfolio was only adn nistered during the fourth nine weeks, it would be difficult 
to determine the patterns that would have resulted if the assessment was used throughout the 
school year as it was designed. 

Based on the results of the three analyses, three conclusions were reached. The first 
conclusion supports statements by Kulm (1990), Cain and Kennedy (1992), and Romberg and 
Wilson (1992) who said that classroom assessment is influenced by the tests that the students are 
mandated to take. The next conclusion is based on the overall and grade 4 through 6 samples 
which indicate that the Stanford Achievement Test and the CTB Mathematics Portfolio assess 
different constructs. Finally, based on the grade 3 and grade 4 through 6 analyses, it appears that 
the constructs assessed by the Stanford Achievement Test change as students move from grade 3 
to grade 4. 
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Table 1 



Correlations Amona the Mathematics Measures for the Overall Sample 









1st 9 


2nd 9 


3rd 9 


4th 9 




SAT 


Portfolio 


weeks 


weeks 


weeks 


weeks 


SAT 


1.0000 












Portfolio 


.12232 

(.19871) 


1.0000 










1st 9 


.18829 


.36831 


1 .0000 








weeks 


(.09519) 


(.00425) 










2nd 9 


.23765 


.38819 


.70698 


1 .0000 






weeks 


(.04827) 


(.00267) 


(.000) 








3rd 9 


.38961 


.03314 


.33196 


.43456 


1.0000 




weeks 


(.00258) 


(.40963) 


(.00926) 


(.00081) 






4th 9 


.14878 


.41850 


.18168 


.28383 


-.12999 


1.0000 


weeks 


(.15124) 


(.00125) 


(.10334) 


(.02289) 


(.18413) 





Table 2 

Unrotated Factor Solution. Communalities and Percent Variance 




for the Overall Sample 










Factor 1 


Factor 2 


communality 


Stanford Achievement Test 


.47921 


-.35451 


.35533 


CTB Mathematics Portfolio 


.60551 


.52706 


.64444 


First 9 Weeks Average 


.80342 


-.03955 


.64705 


Second 9 Weeks Average 


.86567 


- 04803 


.75169 


Third 9 Weeks Average 


.53804 


-.69436 


.77162 


Fourth 9 Weeks Average 


.43880 


.67843 


.65280 


Percent of Variance 


41.2 


22.5 
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Table 3 

Varimax Rotated Factor Loadings for the Overall Sample 



Factor I Factor 2 



Stanford Achievement Test 


.59550 


.02650 


CTB Mathematics Portfolio 


.13832 


.79076 


First 9 Weeks Average 


.64892 


.47534 


Second 9 Weeks Average 


.70262 


.50796 


Third 9 Weeks Average 


.85525 


-.20041 


Fourth 9 Weeks Average 


-.08651 


.80332 



Table 4 



Correlations Amona the Mathematics Measures for Grade 3 Students 









1st 9 


2nd 9 


3rd 9 


4th 9 




SAT 


Portfolio 


weeks 


weeks 


weeks 


weeks 


SAT 


1.0000 












Portfolio 


.33688 

(.06263) 


1.0000 










1st 9 


.44925 


.55158 


1.0000 








weeks 


(.01798) 


(.00389) 










2nd 9 


.21486 


.54997 


.79958 . 


1.0000 






weeks 


(.16846) 


(.004) 


(.000) 








3rd 9 


.44536 


.45464 


.55408 


.50418 


1.0000 




weeks 


(.01890) 


(.01676) 


(.00373) 


(.00836) 






4th 9 


.46328 


.49342 


.54218 


.66850 


.59417 


1.0000 


weeks 


(.01495) 


(.00981) 


(.00457) 


(.00034) 


(.00177) 
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Table 5 

Unrotated Factor Loadings for Grade 3 Students 



Stanford Achievement Test 
CTB Mathematics Portfolio 
First 9 Weeks Average 
Second 9 Weeks Average 
Third 9 Weeks Average 
Fourth 9 Weeks Average 



Factor 1 communality 



.59426 


.35315 


.73112 


.53453 


.85614 


.73297 


.83387 


.69535 


.76773 


.58941 


.82001 


.67242 



Percent of Variance 59.6 



Table 6 



Correlations Amona the Mathematics Measures for Grade 4 through 6 Students 









1st 9 


2nd 9 


3rd 9 


4th 9 




SAT 


Portfolio 


weeks 


weeks 


weeks 


weeks 


SAT 


1.0000 












Portfolio 


-.21516 

(.13577) 


1.0000 










1st 9 


-.01926 


.36896 


1.0000 








weeks 


(.46125) 


(.02667) 










2nd 9 


.28493 


.31952 


.64495 


1.0000 






weeks 


(.07083) 


(.04871) 


(.00011) 








3rd 9 


.36612 


-.36388 


.32484 


.43669 


1.0000 




weeks 


(.02767) 


(.02849) 


(.04584) 


(.01008) 






4th 9 


.02980 


.48252 


42166 


.33218 


-.41034 


1.0000 


weeks 


(.44017) 


(.00465) 


(.01271) 


(.04208) 


(.01505) 
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Table 7 



Unrotated Factor Solution. Communalities and Percent Variance 
for Grade 4 through 6 Students 





Factor 1 


Factor 2 


communality 


Stanford Achievement Test 


.10916 


.60769 


.38121 


CTB Mathematics Portfolio 


.64621 


-.53731 


.70629 


First 9 Weeks Average 


8^246 


.17801 


.74143 


Second 9 Weeks Average 


.81817 


.40644 


83459 


Third 9 Weeks Average 


.11802 


.92239 


.86472 


Fourth 9 Weeks Average 


.68892 


-.45068 


.67772 


Percent of Variance 


38.3 


31.8 





Table 8 

Varimax Rotated Factor Loadinas for Grade 4 Through 6 Students 




Factor 1 


Factor 2 


Stanford Achievement Test 


-.02902 


.61674 


CTB Mathematics Portfolio 


.74970 


-.37978 


First 9 Weeks Average 


78161 


.36129 


Second 9 Weeks Average 


.70701 


.57856 


Third 9 Weeks Average 


- 09052 


.92549 


Fourth 9 Weeks Average 


.77203 


-.28581 
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